{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyPw1gwZ1xfqLolq4oowSNrn",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/sugarforever/LlamaIndex-Tutorials/blob/main/04_Data_Connectors/04_Data_Connectors.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Data connectors"
      ],
      "metadata": {
        "id": "-VjYJcfAkTTu"
      }
    },
    {
      "cell_type": "code",
      "execution_count": null,
      "metadata": {
        "id": "3gyCNIM1Im7-"
      },
      "outputs": [],
      "source": [
        "!pip install -q -U llama-index"
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "import os\n",
        "os.environ['OPENAI_API_KEY'] = 'your valid openai api key'"
      ],
      "metadata": {
        "id": "ZKgOR2HYk-vN"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Use LlamaIndex native data connectors\n",
        "\n",
        "We will the open source langchain tutorial https://github.com/sugarforever/wtf-langchain and load it into LlamaIndex defined documents for query engine usage."
      ],
      "metadata": {
        "id": "-gRIz-JDkWNb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!git clone https://github.com/sugarforever/wtf-langchain.git"
      ],
      "metadata": {
        "id": "p8o-wQXDihsg"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index import SimpleDirectoryReader\n",
        "reader = SimpleDirectoryReader(\n",
        "    input_dir=\"./wtf-langchain\",\n",
        "    required_exts=[\".md\"],\n",
        "    recursive=True\n",
        ")\n",
        "docs = reader.load_data()"
      ],
      "metadata": {
        "id": "X8LnUM2Yiut7"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "docs"
      ],
      "metadata": {
        "id": "TKn4vx29i8x_"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "from llama_index import VectorStoreIndex\n",
        "\n",
        "index = VectorStoreIndex.from_documents(docs)\n",
        "query_engine = index.as_query_engine()\n",
        "response = query_engine.query(\"什么是WTF LangChain？\")\n",
        "print(response)"
      ],
      "metadata": {
        "id": "Qu4lEW_7jJva"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "response = query_engine.query(\"WTF LangChain教程的目的是什么？\")\n",
        "print(response)"
      ],
      "metadata": {
        "id": "GU7NmFnVjoaj"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Load data connectors from LlamaHub\n",
        "\n",
        "We will the open source langchain tutorial https://github.com/sugarforever/wtf-langchain and load it into LlamaIndex defined documents for query engine usage."
      ],
      "metadata": {
        "id": "wl-zzPA9ksay"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from pathlib import Path\n",
        "from llama_index import download_loader\n",
        "\n",
        "MarkdownReader = download_loader(\"MarkdownReader\")\n",
        "\n",
        "loader = MarkdownReader()\n",
        "documents = loader.load_data(file=Path('./wtf-langchain/README.md'))"
      ],
      "metadata": {
        "id": "H30CAo0mk4um"
      },
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "code",
      "source": [
        "index = VectorStoreIndex.from_documents(documents)\n",
        "query_engine = index.as_query_engine()\n",
        "response = query_engine.query(\"LangChain极简入门教程用的什么框架版本？\")\n",
        "print(response)"
      ],
      "metadata": {
        "id": "AqLNKLkrlDkJ"
      },
      "execution_count": null,
      "outputs": []
    }
  ]
}